Stochastic Gradient Descent in Continuous Time

We consider stochastic gradient descent for continuous-time models. Traditional approaches for the statistical estimation of continuous-time models, such as batch optimization, can be impractical for large datasets where observations occur over a long period of time. Stochastic gradient descent provides a computationally efficient method for such statistical learning problems. The stochastic gradient descent algorithm performs an online parameter update in continuous time, with the parameter updates θt satisfying a stochastic differential equation. We prove that limt→∞∇ḡ(θt) = 0 where ḡ is a natural objective function for the estimation of the continuous-time dynamics. The convergence proof leverages ergodicity by using an appropriate Poisson equation to help describe the evolution of the parameters for large times. Numerical analysis of the stochastic gradient descent algorithm is presented for several examples, including the Ornstein-Uhlenbeck process, Burger’s stochastic partial differential equation, and reinforcement learning.